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Abstract 

Supermarket models with different servers become a key in modeling resource man¬ 
agement of stochastic networks, such as, computer networks, manufacturing systems, 
transportation networks and healthcare systems. While the different servers always 
make analysis of such a supermarket model more interesting, difficult and challenging. 

This paper provides a novel method for analyzing the supermarket models with dif¬ 
ferent servers through a multi-dimensional continuous-time Markov reward process. 
Firstly, some utility functions are constructed for the routine selection mechanism 
according to the queue lengths, the service rates, and the probability of individual 
preference. Secondly, applying the state jump points of the continuous-time Markov 
reward process, some segmented stochastic integrals of the random reward function 
are established by means of an event-driven technique. Based on this, the mean of 
the random reward function in a finite time interval is computed, and the mean of the 
discounted random reward function in an infinite time interval can also be calculated. 
Finally, some simulation experiments are given to indicate how the expected queue 
length of each server depends on some key parameters of this supermarket model. 

Keywords: Supermarket model; Routine selection mechanism; Markov reward 
process; Random reward function; Stochastic integral; Event-driven technique. 
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1 Introduction 


Randomized load balancing, where a job is assigned to a server from a small subset of 
randomly chosen servers, is very simple to implement, and can surprisingly deliver better 
performance (for example, reducing collisions, waiting times, and backlogs) in a number of 
applications, such as, data centers, capacity allocation, hash tables, distributed memory 
machines, path selection, and task scheduling. The supermarket model is a dynamic 
randomized load balancing method, and its original idea may be inspired by operation 
mechanism of supermarket checkout in a large supermarket. Because the supermarket 
model has simple operations, quick response, dynamical real-time management, and many 
other advantages, it has been regarded as one of the most effective technologies in the study 
of large-scale stochastic networks with resource management and task scheduling. 

During the last two decades considerable attention has been paid to studying the 
supermarket models through queueing theory as well as Markov processes. Since a simple 
supermarket model was discussed by Mitzenmacher m, Vvedenskaya et al. [H] and 
Turner [42l|43], subsequent papers have been published on this theme, among which, see, 
Vvedenskaya and Suhov [45], Mitzenmacher et al. [32|, Graham mm, Luczak and 
Norris [26], Luczak and McDiarmid [HIES], Brightwell and Luczak [5], Bramson et al. 
[21111], Li and Lui [211122] . Li et al. [23111811201 HQ] a.nd Li [151116] . For the fast Jackson 
networks (or the supermarket networks), readers may refer to Martin and Suhov [29], 
Martin m and Suhov and Vvedenskaya [38]. On the other hand, Janssen m applied 
the discrete-time Markov reward processes as well as the discrete-time Markov decision 
processes to the study of supermarket models with N identical servers. The stability of 
more general supermarket models was discussed by Foss and Chernova [9], Bramson [1] 
and MacPhee et al. [28]. 

There are some successful research on various Markov reward processes, important 
examples include Reibman et al. [M], Ciardo et al. [7], Qureshi and Sanders [33], Telek 
et al. [iQ], de Souza e Silva and Gail [8], Telek and Racz [H], Telek et al. [39], Li and 
Gao m, Stefanov [36], Stenberg et al. m, and two books by Gao [6] and Li [Hi- 

Little work has been done on analysis of the supermarket models with different servers, 
which is more difficult and challenging due to high complexity and percipient subjectivity 
of designing a fair routine selection mechanism with respect to the different servers. Specif¬ 
ically, a practical understanding can indicate that such a routine selection mechanism may 
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depend on the queue lengths, on the service rates, on the probability of individual pref¬ 
erence and so forth. Janssen m described a simple intuitive outline of discussing the 
supermarket model with different servers, and demonstrated that analysis of the super¬ 
market model with different servers will be an interesting and difficult topic in the future 
research. Based on this, Li et al. m provided a birth-death reward process for the 
supermarket model with different servers, and established a system of functional reward 
equations which can be solved by a value iterative algorithm. It is worth noting that 
this paper uses a more general Markov reward process to set up the segmented stochastic 
integrals of the random reward function in the supermarket model with different servers 
by means of an event-driven technique, which is shown to be useful for performance sim¬ 
ulation of a more general large-scale stochastic system. In addition to this, we would like 
to remark two key points: (1) Although the mean-field theory is an effective method in 
the study of supermarket models with the same servers (e.g., see Vvedenskaya et al. [iS] . 
Li et al. |18] and Li and Lui |22])) the complicated routine selection mechanism with 
respect to the different servers makes setting up the systems of mean-field equations more 
difficult. To our best knowledge, up to now no paper has applied the mean-field theory 
to the study of supermarket models with different servers. (2) The generating functions 
are always classical and effective for performance evaluation of many practical stochastic 
systems, but they are not convenient to deal with a multi-dimensional problem, and are 
also very difficult to analyze a system of nonlinear equations. 

The main contributions of this paper are twofold. The first one is to describe a super¬ 
market model with different servers, in which the arrival and service processes are given 
in a detailed discussion, and the reward value at each state is chosen from some practi¬ 
cal points of view. We show that the arrival process of this supermarket model is very 
complicated due to a routine selection mechanism that depends on the queue lengths, on 
the service rates, on the probability of individual preference and so forth. Also, it is seen 
that the routine selection mechanism is very different from that in the supermarket model 
with same servers, where our construction of this routine selection mechanism is based on 
the utility functions so that the subjective behavior of customers is also covered in the 
routine selection mechanism. The second one is to set up a multi-dimensional continuous¬ 
time Markov reward process, and provide a segmented stochastic integral for expressing 
the random reward function in a finite time interval through an event-driven technique. 
Furthermore, we calculate the mean of the discounted reward function in an infinite time 
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interval. Based on this, we give a simple discussion on optimal criterions for designing the 
supermarket model with different servers. Also, we provide some simulation experiments 
to indicate how the expected queue length of each server depends on some key parameters 
of this supermarket model. 

The remainder of this paper is organized as follows. In Section 2, we first describe 
a supermarket model with M different servers. Then we construct a routine selection 
mechanism that depends on the queue lengths, on the service rates, on the probability of 
individual preference and so forth. In Section 3, we set up an M-dimensional continuous¬ 
time Markov reward process, and provide a segmented stochastic integral for expressing 
the random reward function in a finite time interval through an event-driven technique. 
In Section 4, applying the segmented stochastic integral, we compute the mean of the 
random reward function in a finite time interval. In Section 5, we compute the mean of 
the discounted reward function in an infinite time interval. Based on this, we provide two 
optimal criterions for designing the supermarket model with different servers. In Section 
6, we provide some simulation experiments to indicate how the expected queue length of 
each server depends on some key parameters of this supermarket model. Some concluding 
remarks are given in Section 7. 

2 Supermarket Model Description 

In this section, we first describe a supermarket model with M different servers. Then we 
construct a routine selection mechanism that depends on the queue lengths, on the service 
rates, on the probability of individual preference and so forth. 

In the supermarket model, there are M different servers whose waiting rooms are all 
infinite. The service times in each server are i.i.d. and are exponential, and also the service 
rates of the M different servers are denoted as ni, iJ, 2 , ■ ■ ■ respectively. The arrivals 
of customers are a Poisson process with arrival rate A. Because the servers are different, 
it is a key to optimize the service ability of this supermarket model through designing 
a better routine selection mechanism. In fact, designing such a better routine selection 
mechanism will become not only complicated but also subjective due to the difference of 
the M servers. The physical structure of this supermarket model is shown in Figure 1. 

In what follows we will provide a detailed description for how to construct such a 
better routine selection mechanism. Notice that our method for constructing the routine 
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selection mechanism is intuitive and heuristic according to some practical points of view. 



Figure 1: A physical illustration of the supermarket models with different servers 

From Figure 1, it is seen that for the M different servers, each arriving customer joins 
a server (or queue) according to a suitable routine selection mechanism. From a practical 
point of view, each arriving customer chooses one server based on at least three crucial 
factors: (1) Choosing one server with the largest service rate, (2) choosing one server with 
the shortest queue length, and (3) choosing one server with the maximal probability of 
individual preference. 

We write 

X = ( X 1 , X 2 , ... , xm ) , 

which is the vector of the queue lengths in the M servers; 

9 = {91,92, ■ ■ ■ ,gM), 

which is a probability vector of individual preference for choosing one of the M servers. 
In general, the individual preference is based on the priori knownledge, and the present 
feeling etc.; and 

9- = {91,92, ■ ■ ■ ,9m) ■ 

It is worth noting that the two vectors g and g are always inherent in the system, but 
the vector x of queue lengths can change dynamically according to a customer arrival or 
a service completion. 
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Based on the above analysis, let Aj(x) = f{xi,fii,gi) be a routine selection func¬ 
tion which represents the measurement of choosing the ith server for i = 1,2,... ,M, 
where f{xi,Hi,gi) satisfies three conditions; (1) /(xj,//j, (/*) is increasing for Xi > 0, (2) 
f{xi, Hi, gi) is decreasing for gi > 0, and (3) f{xi,gi,gi) is decreasing for gi G (0,1]. 

We assume that if 

iAi(x)}, 

l<i<M 

then the arriving customer joins the ioth server among the M servers. It indicates that 
an arriving customer likes the server with the minimal value in the set of routine selection 
functions 

A = {Ai(x), A2 (x), ..., Am-i{x), Am{x)} . 

From the routine selection function, now we further describe the routine selection 
mechanism as follows: 

The routine selection mechanism: Each arriving customer chooses d > 1 servers 
independently and uniformly at random from the M servers, and joins the server with 
the smallest number in = { Aj^ (x), Ajj (x),..., Aj^_^ (x), Aj^(x)}, where the d selected 
servers are denoted as Servers ii,i 2 , ■ ■ ■ ,id- If there is a tie, servers with the smallest 
number in will be chosen randomly. All customers in any server will be served in the 
first come first service (FCFS) manner. We assume that all the random variables defined 
for the arrival and service processes are independent of each other. 

In what follows we provide some useful interpretation for each element in the set 
A = {Ai (x), A 2 (x),... , A^—]^^x),A^(x)} of routine selection functions. 

Interpretation one: Aj(x) = f{xi,gi,gi) has some usefnl forms 

Note that f{xi,gi,gi) needs to satisfy the above three monotone conditions for each 
element in one of the three vectors x, g and g, thus such a function / : N+ x (0, -t- 00 ) x 
(0,1] ^ can be chosen easily, where N+ = {0,1, 2,...} and = [0, -|-oo). To that 
end, we give some examples to indicate how to construct such a function f{xi,gi,gi) as 
follows: 

(1) A tandem-type decision-making method 

For the three decision variables Xj, gi and gi, we set up a tandem-type decision-making 

structure as x,- • — • —, thus it is seen from a normalization that the routine selection function 
* Mi Si ’ 
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is given by 


Aj(x) = 


1 + 


(J'idi 


M r 

E 

i=i '■ 


= 1,2,...,M. 


1 + 


H9j 


(2) A weighted-type decision-making method 

For the three decision variables x*, /r* and gi, we take a weighted-type decision-making 

structure as /3iXj -|- /? 2 — + /?3 —, where the weighted coefficients satisfy that > 0 and 

9i 

/3i + /32 + /^s = 1, thus the routine selection function is given by 


Ai(x) 


1 -|- j3iXi + 1^2^^+ 

M r 


E 

1=1 


1 -h I3ixj + 132^ -h / 33 A 


fij 


‘9j 


1 = 1,2,... ,M. 


Interpretation two: There exist multiple minimal elements in 

For Ad = {Aij(x), Ai 2 (x ),..., Ai^_^{x), Ai^{x)}, set 

3?min (d) = Oo : Aio(x) = min {A 4 (x)} 

l<k<d 

Then we have two cases: (1) 3?min {d) contains only one element, and (2) 3?min (d) contains 
multiple elements. For the former, the routine selection of the arriving customer is simple 
for choosing Server while for the latter, the routine selection of the arriving customer 
has a little complicated, for example, a simple mode is taken as that if there is a tie, servers 
with the smallest number in 3?min (d) will be chosen randomly, e.g., see Vvedenskaya et al. 
|44j and Mitzenmacher m- 

To use more information in the set 3?min {d ), we may set up some new routine selection 
ways. If there is a tie (that is, 3?min (d) contains multiple elements), then servers with the 
smallest number in 3?min (d) may be chosen by means of other ways, for example, either 

(1) for all the different elements in 3?min (d), the arriving customer joins the server with 
the biggest service rate; 

(2) for all the different elements in 3?min {d), the arriving customer joins the server with 
the shortest queue length; 

(3) for all the different elements in 3?min (d), the arriving customer joins the server with 
the maximal probability of individual preference; or 

(4) some hybrid combination from the above (1), (2) and (3). 

In this paper, we will not discuss the above four cases, which are interesting and will 
be studied in our future work. 
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Interpretation three: Usefnl relations between onr above model and the 
ordinary supermarket model 

On the one hand, when /Ui = /i 2 = ■ ■ ■ = ^ and gi = §2 = ■■■ = 9 m = h is 

seen that 

= f{xi,Hi,gi) = f 

which shows that the routine selection of the arriving customer only depends on the vector 
X = {xi,X 2 , ■ ■ ■ ,xm), hence the arriving customer joins the server with the shortest queue 
length, e.g., see Vvedenskaya et al. [H]. On the other hand, we remark that the probability 
vector g = { 91 , 92 , ■■ ■ ,gM) of individual preference can give rise to the study of modern 
supermarket business or network economy. 

In the supermarket model with different servers, data collection and analysis is also 
a key task. Therefore, it is interesting that the routine selection mechanism can be de¬ 
signed from a data-based practical point of view. This will motivate statistical analysis of 
supermarket models with different servers from many real areas. 

3 A Markov Reward Process 

In this section, we set up an M-dimensional continuous-time Markov reward process, and 
provide a segmented stochastic integral for expressing the random reward function in a 
finite time interval through an event-driven technique. 

In order to set up a continuous-time Markov reward process, we need to discuss the 
arrival and service processes, both of which lead to the state jumps of this Markov re¬ 
ward process. At the same time, we choose a suitable reward value at each state in this 
supermarket model. 

(1) Analysis of the arrival processes 

In this supermarket model, the arrival process of customers is a Poisson process with 
arrival rate A. Each arriving customer chooses d servers independently and uniformly at 
random from the M servers, and joins one server with the smallest number in the set 
Arf = Ai 2 (x),..., Aj^_j(x), Aj^(a:)}. If there is a tie, servers with the smallest 

number in the set A^ will be chosen randomly. 

In order to express the routine selection mechanism of each arriving customer, we need 



to introduce an ascending function a : [0,1]^ ^ [0,1]^ as follows: 

a = (Afc^(x),Afc2(x),...,Afc^(x)) 

for = (Ai(x), A 2 (x), . .., Am{x)), where 

0 < Afc^(x) < Afc2(x) < ... < Ak^{x) < 1. (1) 

For the ascending function a (A^*^), it is necessary to explain the order numbers ki 
for 1 < i < M. Note that ki denotes the kith, element of the original order number vector 
A(^). For example, if A^^^ = (1/3,1/2,1/6), then a (A^^^) = (1/6,1/3,1/2). It is obvious 
that Afcj(x) = 1/6 and ki = 3; Ak2{x) = 1/3 and k2 = 1; and Ak^{x) = 1/2 and k^ = 2. In 
general, for these order numbers before and after sorting, we provide their corresponding 
relation in Figure 2. 



The order number in The order number in 


Figure 2: The order relation before and after sorting the M servers 

Based on the ascending function with the sorting process, it is a key how to describe 
the arrivals of customers at each server in this supermarket model. It is worthwhile to note 
that Janssen m gave an effective method for analyzing the ascending function as well as 
the arrival processes at the M different servers. Here, we provide a detailed description 
for the Janssen’s method as follows. 

For a sorted vector x with 0 < xi < X 2 < • • • < xm, it follows from (3.6) and (3.7) 
in Janssen m that the probability that the arriving customer first randomly selects d 
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servers from M servers, and then enters the zth server (that is, the zth shortest queue is 
also in the d selected servers) is given by 


k{M, i, d) 


1 {M-i)\{M-d)\ 

^ (M-i-d+l)\{M)\ ’ 

0 , 


I <i < M - d + l, 
M-d + 2<i<M, 


( 2 ) 


and specifically, we may randomly give a sort for these servers whose queue lengths are 
equal. At the same time, Lemma 3.2.1 in Janssen m proved that for 1 < d < M, 

M M-d+1 

Y^k{M,i,d)= ^ k{M,i,d) = l. (3) 

i=l i=l 

Now, we explain the probability k{M,i,d) for sorted vector x with 0 < xi < X 2 < 

• • • < XM- 

As seen from Figure 3, notice that the arriving customer first randomly selects d servers 
from the M servers, and enters one server with the shortest queue length among the d 
selected servers (if there is a tie, then servers with the shortest queue length will be chosen 
randomly), thus the routine selection mechanism is converted to the probability k{M, i, d) 
of entering the ith server for 1 <i < M. Therefore, Xk{M, i, d) is the arrival rate that the 
customers arrive at the server with the fth shortest queue length among the M servers. 



Before sorting 

The order numbers of servers are original 


An arrival 


\k{MXd) 



An arrival 


\k {M,2,d) 



An arrival 
-► 

Xk(M.M-l,d) 



An arrival 


Xk(M,M,d) 



After sorting by 0 < JC, < < • • ■ < 

The order numbers of servers are rewritten 


Figure 3: Some interpretation on the probability k{M, i, d) 

For the ascending function cr (A^^^) which is similar to the sorted vector x with 0 < 
xi < X 2 < ■ ■ ■ < xmi it is easy to see that the Janssen’s method still work. Thus, for the 
ith element in a (that is, the kith element in A^’^^ this corresponds to the fcjth 
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server in this supermarket model), using ([3|) we obtain 


k{M, ki, d) 


j (M-kiy.iM-dy. 

“(M-A:i-d+l)!(M)!’ 

0 , 


l<ki<M-d+l, 
M-d + 2<ki<M. 


Obviously, we also have 


M 


M-d+l 


''^^k{M,ki,d) = k{M,ki,d) = l. 


According to the probability k{M, ki,d), it is clear that the arrivals of customers at the 
kith, server is a Poisson process with arrival rate Xk(M, ki,d) for i = 1,2,..., M. Hence, 
the Poisson arrival rate at the fcjth server is given by 



l<ki<M-d+l, 
M - d + 2 <ki< M. 


(4) 


(2) Analysis of the service processes 

Analysis of the service processes is simpler than that of the above arrival processes in 
this supermarket model. Let be an indicator function of the event: {xi > 0}, that 

is, 



1 , Xi > 0 , 

0 , Xi = 0 . 


The service rate of the ith server may be written as ^il^xi>o}^ because the server is idle 
when there is no customer (i.e., Xi = 0) in this server. 

(3) Choosing a suitable reward value at each state 

Note that Afc^(x) > Afc^(x), it is obvious that if the value [Afc^(x) — Afc^(x)] 
is bigger, then the customers in the M servers are not distributed well. On the contrary, if 
the value — Afc^(x)] /Afc^(x) is smaller, then the customers in the M servers are 

load balanced very well. Thus, our purpose of designing and optimizing this supermarket 
model is to make the value [Afcj^^(x) — Afc^(x)] /Afc^(x) as small as possible. At the same 
time, it is easy to see that 


mm 


{^kMix)} 


max 


{Afci(x)} < Akj^{x) - Afci(x). 


Based on the above analysis, we may choose two different reward values at state x as 
follows: 


^min(3:) .— A^j^ (x) 


(5) 
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and 

l^ma.xix) '■= A.kMi^)- (®) 

Notice that we use the two reward values: rmin(a:^) and rmax(3:^), to be able to provide a 
better observation on performance of this supermarket model, which will be studied in 
Subsection 5.2. 

In the remainder of this section, we introduce a useful continuous-time Markov process, 
which will be used to give performance computation and performance simulation in the 
supermarket model with different servers. 

Let Xfc (t) be the number of customers in the feth server of this supermarket model at 
time t > 0, and 

X(t) = (Xi 

Obviously, {X (t) : t > 0} is an M-dimensional continuous-time Markov process on the 
space state = {x = (xi, X 2 ,..., xm) ■ x^ > 0,1 < k < M}. 

Let r (x) be a real function for x G O, and r (x) denote a reward value of this Markov 
process {X (t) : t > 0} at state x. Based on this, we define a random reward function as 

$(t)= rr(X(0)d^, (7) 

Jo 

which is a stochastic integral, e.g., see Chapter 10 in Li [T^] for more details. 

In what follows we propose an event-driven technique to deal with the random reward 
function (t). To this end, we denote by ryi, 772 , r/ 3 ,..., the n successive state jump 
points of the Markov process {X (t) : t > 0} in the finite time interval [0, t], it is clear that 

0 < r/i < r/2 < • • • < i < Vn+i- (8) 

Note that rjk = rj'^, and % is a state jump time of Markov process {X (t) : t > 0}, thus 
it is helpful for understanding the stochastic integral J'Qr(X(^))d^ under an interval 
decomposition as follows: 

[0, t] = [0, 7]^) U [ryi, 772 ') u [ 772 ,%)•••□ [r]n-i,r]~) U [ry^, t], 
it follows from ([7]) and ([8]) that 

ch (t) = r r (X (0) d^ + Yl ^ (X ( 0 ) d^+ t r (X (0) d^, (9) 

which is a segmented stochastic integral for expressing the random reward function <I> (t). 
Note that this segmented stochastic integrals will be useful in our later study. 
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4 Computation of the Expected Reward Function 

In this section, we use an event-driven technique to compute the mean of the random 
reward function in a finite time interval, where our computation is based on the above 
segmented stochastic integral, which is expressed through the successive state jump points 
generated by either customer arrivals or service completions. 

From (a) in Figure 4, let {Af {t) : t > 0} be a Poisson process with parameter u = 
X + Hi + ^2 + ••• + Hm- Then for /c > 0 

Pfc (t) = P{Ar(t) = A:} = e-*^. 



(a) 

State transitions from state X 


Figure 4: State transitions and associated rates at State x 

We assume that the random sequence {Y^ : fc > 1} is i.i.d. and is exponential with 
mean l/o;. Let r]n = Then M (t) = sup{n : !]„ < t}, and 0 < rji < r ]2 < ■ ■ ■ < 

rjn <t < rjn+i- From Section 2.3 in Ross [35], it is easy to see that 

P{Vi<s\Af{t) = l} = P{Yi<s\Af{t) = l} = j. 

Let the n-dimensional probability distribution be 

F{si,S 2 , ■ ■ . ,Sn) = P {m < Si) ^2 < ^2, . . . < Sn} 
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and the n-dimensional probability density function 

dn 

f ('Sl, ■52, • • • , Sn) = F (si, S2i • • • , Sn) ■ 

Then it follows from Theorem 2.3.1 in Ross |35| that 

Tl\ 

f (si, S2, ■ ■ ■ ,Sn) = —, 0 < Si < S 2 < ■ ■ ■ < Sn < t. 

V'‘ 

At the same time, Theorem 2.3.1 in Ross [35] demonstrates that given that N (t) = n, the n 
arrival times 771 , 772 ,.. ■, have the same distribution as the order statistics corresponding 
to n independent random variables uniformly distributed on the interval (0,t). Thus, 
using the condition: 0 < r]i < rj 2 < ■ ■ ■ < r]n < t, we obtain 

E [rji] = E[r] 2 -rji] = ■■■ = E[r]n- rjn-i] = E[t-rjn]= ^ ■ (10) 

n + 1 

R is seen from (a) and (b) in Figure 4 that for k > 1, the Markov process {X (t) : t > 0} 
transits to State X (rjk) from State X {rj^) (i-e., a state jump), where State X (ry^) may be 
either State X (77^) — ej due to a service completion by Server j for 1 < j < M, or State 
X (77^) + Cfc. due to a customer arrival at Server fcj with the routine selection mechanism 
for 1 < 7 < M. Note that X (77)”) = X (0) = x and X (77^) = X (rjk-i) for 2 < k < n, 
thus we have 


X (77fc) G {X ( 77 ^ ) - Cj : 1 < j < M} U {X ( 77 ^ ) + Cfci : 1 < 7 < M} 

= {X ( 77 fc_i) - ej : 1 < j < M} U {X ( 77 ^. 1 ) + : 1 < 7 < M} . 


Let An be the 77 th inter-arrival time of the Poisson process with arrival rate A, and 
Sn ' the exponential service time with service rate Hk of the 77 th customer in Server k. 
Then {A„} and are all i.i.d for 1 < k < M. In this case, we write that A = Ai and 

5(^1 = for 1 < fc < M. Based on these random variables A and for 1 < A: < M, 
we can express the random events of the Markov process {X (t) : t > 0} at time 77 ^ as 
follows: 

(1) An arrival at time rjk 

In this case, we need the sufficient condition 


It is easy to compute that 


mm < D" " > 
l<k<M I J 


a = P { A < min 

l<k<M 




A 


A + /7l + /72 + • • • + fJ-M 
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(2) A service completion in Server j for 1 < j < M 
In this case, we need the sufficient condition 


< 


mm 




min 

l<k<M 



We can that 


bU) 


= P 


sU) 


< 


min 



min 

l<k<M 



_/At_ 

A + + /r2 + • • • + ^^M 


Now, we compute the conditional mean Ex [<h (/)], where Ex [•] = -B [• | X (0) = x]. 
We have 


E [^> (t) I X (0) = x] = ^ P {AA (t) = n}E[^ (t) | X (0) = x, W (t) = n] 

n=0 


=P {M (t) = 0} E [<^ (t) I X(0) =x,A'(i) = 0] 

oo 

+ ^ P {AA (t) = n} B [^> (t) I X (0) = X, AA (i) = n], (11) 

n=l 

Since Af (/) = 

B (t) I X (0) 


0, it is clear that r]i > t, this gives 


= x,Af (t) = 0] = E 


rr(X(0)de|X(0) = x,7?i>t 


= r (x) t. (12) 


For n > 1, notice that the event {Af (t) = n} is the same as the event {0<T/i<r72<--- 
< r]n < t < rjn+i}, thus we obtain 


B[$(t) I X(0) =x,Af{t) = n] 


= E 


= E 


[ r (X (^)) d^ I X (0) = X, 0 < r/i < 7?2 < • • • < ?/n < t < Vn+i 

Jo 


ni 


r (X (^)) d| I X (0) = X, 0 < 7/1 < T/2 < • • • < < / < Vn+i 


n—1 

+ E® 


k=l 


nk+1 


r (X (^)) dC I X (0) = X, 0 < 7/1 < 7/2 < • • • < T/„ < t < r]n+i 


'Vk 


+ E 


r (X (^)) d^ I X (0) = X, 0 < 7/1 < 7/2 < • • • < 7/„ < t < r/„+i 


L*/ r]n 


(13) 


To compute (1131) . we may observe some useful relations as follows: 

(1) If ^ € [0,7/^) and X (0) = x, then X (^) = x for ^ G [0,7/j"). 

(2) For 1 < j < n- 1, if ^ G and X (t/^) = y, then X (^ = y for ^ G [t/^, 7/"^J. 
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(3) If ^ € [r]n, t] and X (r/„) = 2;, then X (^) = 2; for ^ G [??n, t]- 
Based on the above useful relations, together with (|10l) . we obtain 


fVi 

/ r (X (^)) d^ I X (0) = X, 0 < r/i < r/2 < • • • < ??n < t < ?/n+i 
Jo 

= r (x) E [iJi] = r (x) E [r]i] = r (x) 
for 1 < A; < n — 1 


t 

n + 1 ’ 


E 


Hk+i 

/ r (X (^)) d^ I X (0) = X, 0 < r/i < r/2 < ■ ■ ■ < r?n < A < rjn+i 
Jvk 


= E[r(X (r/fc)) I X (0) = X, 0 < r/i < r/2 < ■ ■ ■ < ??n < A < Vn+i] ' E - rfu] 
= E[r{X {r]k)) I X (0) = X, 0 < r/i < r/2 < ■ ■ ■ < ??n < A < Vn+i] ■ E [r]k+i - Vk] 

f 

-E [r (X (rik)) I X ( 0 ) = X, 0 < T/i < 772 < • • • < < t < ry^+i] 


n + 1 


and 


E 


r (X (0) d^ I X (0) = X, 0 < 771 < ?72 < • ■ ■ < ?/n < A < Vn+i 




= E[r(X {rjn)) I X (0) = X, 0 < 771 < 772 < • • • < 77„ < t < 77^+1] ■ E[t- r]n] 
t 


77+1 

We write that for 1 < A: < 77 


E [r (X {r]n)) I X (0) = X, 0 < 771 < 772 < • • • < 77„ < t < 77^+1]. 


^k = E[r (X (77fc)) I X ( 0 ) = X, 0 < 771 < 772 < ■ ■ ■ < 77„ < A < 77^+1] . 

It follows from ifTTI) . (fT^ and (fT 3 l) that 

00 / I \ 77. I 

E [d> (A) I X ( 0 ) = x] = r (x) ^ - - 

n=l 

Clearly, it is a key to compute the functions: 3 ?^ for 1 < A: < 77. 

Now, we use (|14p to compute the conditional mean E [<I> (A) | X ( 0 ) = x] of the random 
reward function d? (A) through an event-driven technique. To this end, our computation is 
decomposed in the following three steps: 

Step one: Compute = E[r (X (771)) | X ( 0 ) = x, 0 < 771 < 772 < • • • < 77^ < A < 77^+1] 
It is seen from (a) and (b) in Figure 4 that the Markov process {X (A) : A > 0 } transits 
to a state X (771) from the initial state x, where the state X (771) may be either State x — ej 


r{x) + '^ 'Slk 

k=l 


(14) 
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due to a service completion by Server j for 1 < j < M, or State x + e^. due to a customer 
arrival at Server ki for 1 < i < M. Using the routine selection mechanism, we have 


X (r/i) € {x — Cj : 1 < j < M} U {x + Cfcj : 1 < i < M} . 

From (a) and (b) in Figure 4, it is seen that the computation of 3?i is decomposed into 

two parts: One by an arrival, and another by a service completion. Thus we obtain 

M M 

3?i = (x + CfcJ - a- k{M,ki,d) + ^r{x - Cj) ■ (15) 

i=l j=l 

where a ■ k {M,ki,d) is the probability that an arriving customer joins Server ki, and 
■ > 0 } is the probability that a service is completed in Server j. 

Step two: Compute ^2 = E[r (X (772)) I X ( 0 ) = X, 0 < r/i < 772 < • • • < 7 ?n < t < dn+l] 
It is seen from (a) and (b) in Figure 4 that the Markov process {X (t) : t > 0 } transits 
to a state X (772) from a state X (771) in the set 


{x — ej : 1 < j < M} LI {x + eki : 1 < 7 < M} , 


hence we have 


X 6j T 


X(7?2) 


x + eki+ , 

x — Cj — ei 


{ x + Cki-ei 


thus we have 


if an arrival occurs in Server km at time 772, 
if an arrival occurs in Server km at time 772, 
if a service is completed in Server I at time 772, 
if a service is completed in Server I at time 772, 


X ( 772 ) G {x - Cj + : 1 < J, 777 < M} U {x + Cfc. + : 1 < i, 777 < M} 

U {x — ej — ei : 1 < j, I < M} U {x + — e; : 1 <i,l < M} . 

Based on the above analysis, it is seen from (a) and (b) in Figure 4 that 

{ M M 

^ (x - Cj + CkJ) ■ ■ ak{M, km,d) 

m=l j=l 

MM ^ 

+ EE r (x + + Skm) ' ki,d) ■ cik {M, kmi d) / 

m=l i=l ) 

{ M M 

Y^Y^r{x-ej- ei) ■ 

7=1 j=l 

MM 'I 

+ EE^(- + efc, -ez) \ . 


( 16 ) 


7=1 j=l 
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Table 1: The order number of servers with either an arrival or a service completion 


State jump points 

Server number by arrival 

Server number by service 

m 

kii 

h 

m 

h- 

^12 

h 




Tin 

ki 

jn 


Step three: Compute = E[r (X (rjk)) | X (0) = x, 0 < r/i < 772 < • • • < t < Vn+i] 
for 3 < A: < n 

From the above two special computations, here we will further develop the event-driven 
technique to calculate the conditional mean of the random reward function. 

For the general term 3?^, our computation is more complicated than that in the above 
two special cases. To that end, we need to introduce some notation to record the order 
number of the server with either an arrival or a service completion at each of the state 
jump points rj^ for k = 1,2,... ,n. Observing the two expressions (fTSll and (fT6|) . the order 
numbers of the servers need to relate to the state jump points % for k = 1,2,... ,n. For 
simplicity of description, it is necessary to list some notation in Table 1, the purpose of 
which is to express the state jump points and associated useful information. 

For simplification of description, when deriving some conditional means involved below, 
we introduce a convention notation: Ey {X\ = E\E [X|y]] (that is, a deterministic value), 
where X and Y are two random variables. 

From Steps one and two, it is easy to see that 3 ?^ depends on the k successive samples 
for the states X (r/m) for m = 1 , 2 ,... , A: — 1 . To describe the states X (77^), we express the 
successive state jumps as follows: X ( 0 ) x X (771) x X( 772 ) X • • • X X (77fc_i), where Ax B 
denote the Cartesian product from the set A to the set B. Since X (77^) = X (77fc_i) 
and our computation depends on the A; — 1 successive samples for the states X (77m) for 
777, = 1 , 2 ,... , A; — 1 , we set X (77^7) = X ( 0 ) x X (771) x X (772) x • • • x X {r]k_i), hence the 
first A; — 1 samples X ( 0 ) x X (771) x X( 7 ? 2 ) X • • • X X (77^-1) is used to record our previous 
computational process. Therefore, we obtain 


— -®X{0)xX{r)i)xX(»?2)x-xX{r;fc_i) [>’ (X (%))] ■ 
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In this case, we need to represent the initial state X (0) x X (r/i) x X (r/ 2 ) x • • • x X {rjk-i) by 
means of X {r]m) G {• — : 1 < jm < Af} U {• + : 1 < < M} for 1 < m < A: — 1, 

thus we have 

X (0) X X (r/i) X X im) X • • • X X (%_i) G 

where 

0(^-1) = 00 X 01 X 02 X • • • X 0fc_i, 


00 = {x} 

©! = {•- eji : 1 < ii < M} U |« + : 1 < ii < m| , 

©2 = {• - ej2 : 1 < i2 < M} U |« + Cki^ : 1 < ^2 < m| , 

©fc_i = {• - : 1 < jk-i < M} U |« + eki^__^ : 1 < ik-i < m| . 

To understand the elements in the set we need the Cartesian product as follows: 

{A, B} X {C,D} = {Ax C,Ax D,B X C,B X D}, 

where A, B, C, D are four sets with finite elements. 

In the set 01*^“^), the k elements are successively taken from the subsets 0o, ©1, ©2, • • •, 
©fc_i, for example, x G ©0, • - ej^ G ©1, • + G ©2, • • •, • - G ©fc-2, • + G 

©fc_i. For the successive k elements, we have a simple computation through the following 
convention 

{x} {• - ejJ = X - ej,, 

{x} {• - Cj J I* + ek^^ } = X - Cji + ek^^ , 

{x} {• - Cj J |. + |. + I = X - Cji + - ej,_^ + . 

Based on this, we can easily give a sample of the initial state X (0) x X(7?i) X X(7?2) X 

—¥ —^ ^ 

• • • X X {'r]k-i) in the set 0(*^“i). 

—> 

Now, we compute the conditional mean F;x(o)xX(r,i)xX( 772 ) x-xX0fe_i) [r (X {rjk))] by 
means of an iterative algorithm as follows: 
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(a) For m = 1, we have 


£x,0)[r(XO.))l = E ak (M, ki, d)r{x + + ^b^^h{^.>o}r {x - ej). 


(b) For m = 2, we have 

{ M M 

'^'^r{x - Cj + ekjj ■ • ak (M, d) 

m=l j=l 
M M 

+ EE r (x + 6/^^ + Gf^m) * (A^, ki^d) ‘ dk {M', 

m=l 2=1 


[ «=1 i=l 
M M 

+ EE^(- + efci - eO • a/c (M, /cj, d) • 
1=1 i=l 


(c) For m = k > 3, we take an element t/k-i G then 


IVl 

[OX (%))]= yfc_i + ^ / ivk-i) ■ ak (M, A:^, d) 


M 

+ ^ r {yk-i - ei) f (yk-i) ■ > 

1=1 


and / (yk-i) is the probability that the state yk-i occurs. It is necessary to provide some 
interpretation for the probability / (yk-i) by means of the following three examples: 

(c-1) / (yo) = 1 due to yo = x. 

(c-2) If 2/1 = X - Cjj, then f (yi) = If yi = x + , then /(yi) = 

ak (M, ki -^, d). 

(c-3) If y 2 = X - Cji - Cj^, then / (ys) = if ya = 

X + efc. — 6 ^ 2 , then /(y 2 ) = ak{M,ki^,d)h^^'^^\r ^ ; and the other two can 

similarly be computed and both of them are omitted here. 
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Note that yk-i € using (HZD we obtain 


-Ex{0)xX(7?i)xX(772)x-xX(r;fc_i) [r (X {tjk))] 


= E 


M 

^ r + efc,^ ^ / (yfc-i) • ak (M, , d) 


M 

+ '^r{yk-i -ei)f{yk-i) 

i=i 


6«1 


{(3/fe)i>0} 


(18) 


Now, we further discuss the key computation of / (j/fc-i) whose purpose is to provide 
some new highlight on the calculation program. 

Intuitively, the set of jump states: X ( 771 ) x X ( 772 ) x • • • x X {rjk), can be decomposed 
into two subsets: One for an arrival and another for a service completion. Based on this, 
we record the order numbers for either the arrivals or the service completions, for example, 
if X(r7m) occurs at an arrival, then we record the order number as Vm] while if X.{rjm) 
occurs at a service completion, then we record the order number as Wm- Therefore, the 
set of the order numbers is given by 


{1,2,3, ...,k} = 1 ^* 3 , 


K:, 








where 0 < p < k. Specifically, if p = 0, then the set of the order numbers only contains 
the service completions; while if p = fc, then the set of the order numbers only contains 
the arrivals. 

Based on the two subsets { Vi^, Vi ^} and {, Wj ^, Wj ^, • • •, ^jk-p }) 
obtain 


aP n k{M,k 


m=l 




k-p ,. 

,d)-n 

h=l 



JWrr 


h E 

Vs<Wf^- 


eu. 

.1 vs 



7 p 

m=l 


k—p 



h=l 


0 

where we have some convention on II • 

m=l 


0 

1 and X) • = 0) notice that kiy, depends 

_, im 
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on the state Thus we obtain 


-®X{0) xX(»7i) xX{r; 2 ) x - xX(%_i) [r (%))] 

k M MM M ^ 

E E ■■■ E E ■■■ E “TT**"-*--''.™''*) 


P=0iVi=i ivi=ljwi=^ iiVfe-p=l 


h=l 


h-i 

E efe. 

m=l Vs<W^^ —1 


/ p fc-p \ 

•r -&wj- (19) 


>0 


m=l 


^=1 


JWh 


Similarly, from the two subsets { Vi^ ..., Vi^ } and { Wj ^, Wj ^, Wj^,... 

we obtain 


fiVk) = aP IlkiM,ki^^ ,d) • 

m=l '*'m h=l I 


h-i 
X- J2 e. 


'Wm E fife, 

m=l Vs<W;j-l 




>0 




In the remainder of this section, we finally compute the conditional mean E [<I> (t) | X (0) = x] 
of the stochastic integral <I> {t) according to the above steps one to three. 

It follows from (fTT)) that 

E [$ (t) I X (0) = x] = r (x) + V -— 

^ ^ n! n + l 

n=l 

X < r (x) + ^ ilix(0)xX(7?i)xX(»?2)x-xX(r7fc_i) (X (%))]!• ) (20) 

I fc=l ^ ^ ^ ^ J 

where ^x(o)xX( 7 ?i)xX(,? 2 )x-xX(%_i) [’’(X(%))] is given in ([H]) or ([l9]). 


5 A Markov Discounted Reward Process 

In this section, we provide an effective method for computing the mean of the discounted 
random reward function in an infinite time interval. Based on this, we give a simple 
discussion on optimal criterions for designing the supermarket model with different servers. 

In the infinite time interval [0,+oo), it is possible that [<h (+oo) | X (0) = x] = 
+ 00 . To avoid the infinite case, the random reward function is always taken as a dis¬ 
counted reward. Notice that r (x) is a reward value of the M-dimensional Markov process 
{X(t):t > 0} at state x G 11, we define a discounted random reward function as 

r+oo 

T(/3)=/ e-^V(X(t))dt, (21) 

Jo 
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where /3 > 0 is a discounted rate, and the discounted factor guarantees that 'h (/3) is 
finite a.s.. 

If 'h (0) is finite a.s., then ^ (0) = E [$ (+oo)] is an ordinary (non-discounted) random 
reward function, as studied in Section 4 with t +oo. 

Now, we provide a segmented stochastic integral for expressing the random reward 
function d/ (/3), this will be useful in our following computation. 

Let r]i,r] 2 ,rj 3 ,... be the successive state jump points of the M-dimensional Markov pro¬ 
cess {X (t) : t > 0} in the time interval [0, -|-oo), it is clear from the Poisson or exponential 
assumptions that 

0 < r/i < r/2 <%<•••■ 


At the same time, the sequence: r]i, ?/n+i — Vn for n > 1, is i.d.d. and exponential with 
mean 1/cu. Note that the case with the time interval [0,-|-oo) is different from that in 
Section 4 with respect to analysis of the uniform distributions. 

Note that 

[0, +oo) = [0, r]-)\j[r]i,r]-)U [m,V 3 ) U • • • , 
it follows from (j2ip that 


^ iP) = / (X (f)) df-I-^ (X (f)) df. 

Jo Jr]j 

Thus we obtain 
E[^{/3) I X(0) =x] = 


( 22 ) 


+ E® 


e-^V(X {t))dt 


= r(x) E^ 


P" e-J^r(X{t))dt 
Jo 

r\-J^dt] +f;i^x(.-) \rim) 

Jo Jm 


(23) 


Note that our following computation shows that E [T (/3) | X (0) = x] is not about the 
taken sequence {rj^ : /c > 1}. 

Since r (X (t)) = r (x) for t € [0,7?j") and r (X (f)) = r (r/^) for t G [i]k,i]k+i)^ need 


to compute E 


e ^^dt and E e ^^dt 


It is easy to check that 


E 


rvi 

Jo 


-J^dt 


for A: > 1. 


1 


/3-|-A-|-/ri-|-/r2 + ‘‘‘ + /^M 


(24) 
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To compute E 


e , let the random variable T be exponential with parameter 


A + /ii + /U 2 + • • • + fJ-M ■ Then we have 


E 

Hk+i „ 

/ e-O^dt 

= 


1- 

1- 

•40 

1 

+ 

-se 


Jyk 



-Jvk J- 


(25) 


where [•] denote such a mean with respect to the random variable Y. It is clear that 
r]k is a random variable with the Erlang distribution of order k as follows: 


k-l 

< ?/} = 1 - exp {- (A + /ii + /r 2 H-h ^ 1 M) y} ^ 

3=0 


[(A + + /i2 + • • • + k-Ad) vY 

J'- 


Hence it follows from (j25p that 


E 


Hk+i 

Jvk 




/•+00 r+oo py+X 

/ / / e~^^dtdP {r < x} dP {r]k < y} ■ 

Jo Jo Jy 


(26) 


Based on (|2^ and (f26]l . we set 


and for k > 1 


00 m = E 


0k (/3) = E 


rvi 

Jo 


f^k+i 


'Vk 




-^^dt 


Note that the sequence {0n (/?) : n > 0} can explicitly be determined by ([25]) and 
although we omit some computational details. 

It is easy to check that 


E 


X{0) 


fVi 


e-^V(X(t))dt 


r (x) 


/? + A + /il+/i2 + ''' + k-M 


= Oq (/3) r (x) 


Now, we compute Px(?j“) 
follows: 


e (X (t)) dt by a similar method given in (IlSp as 


^X(0)xX(7?i)xX(j72)x-xX(r?fc_i) 

M 


f%+i 
'vk 


e-^V (X {t))dt 
^ r {yk-i + ) / (yfc-i) • ak (M, ki ^^, d) 


--Ok m Y. 

|_4=i 
M 

+ Y^ (yfc-i - ei) f (yk-i) ■ 


1=1 


(27) 
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It follows from ([23t) . (fMll and (l271) that 


i^['I/(/3) |X(O)=x] = 0o(/3)r(x) + ^0fc(/3) 

fc=i j/fe_iGe(''-i) 


M 

^ r + efe,J / ivk-i) 


‘^k — 1 


M 


xak{M,kii^,d) + '^r {yk-i - ei) f (y^-i) ■ l{{j/fe),>o} 


i=i 


(28) 


It is seen from (l28l) that E ['h (/3) | X (0) = x] is disconnted by the /3-seqnence {6n {(3) '■ n > 0}, 
which guarantees that E ['h (/3) | X (0) = x] < +oo. 

In the remainder of this section, we provide a simple discussion for optimal design 
of the supermarket model with different servers. Specifically, such an optimization may 
be realized through an event-driven technique with performance simulation as well as 
perturbation realization, e.g., see Cao [6] and Xia and Cao [36]. 

To realize an optimal design, the parameters of this supermarket model can be classified 
as three different groups: (I) The customer arrival parameters A; and gi,g 2 , ■ ■ ■ ,gM- (2) 
The customer service parameters M] d] yi, fj, 2 , ■ ■ ■, (3) The economic parameters r (x) 

for X G n. In general, the customer arrival parameters are always fixed, given that the 
customer resource and environment are fixed; while the economic parameters are chosen in 
order that performance optimization of this supermarket model can be easy to be carried 
out. Based on this, our optimal design is to focus on taking the optimal service parameters: 

M; d] ki,k2, ■ ■ ■, kM- 

From a practical point of view of performance optimization, we take two different 
reward values: rmin(a^) := Afc^(x), and r^g^^{x) := Akj^{x) for x G II, respectively. Thus, 
for r (x) = rmin(a^) = Afc^(x) for x G H, we write 


(/3, r„,in) = E[^ (/3, rmin) I X (0) = x] ; 


while for r (x) = rmax(aj) = ^kiai^) x G H, we set 

^ (/?, W) = E[^ (/3, w) I X (0) = x]. 

Based on E [T (/3) | X (0) = x], using an event-driven technique with performance sim¬ 
ulation as well as perturbation realization, we can obtain the optimal decision parameters 
M* ; d*‘, ki, k 2 ^ ■ ■ ■ ^ k*M such that 

’I'* (/3,rmin) = maxl^f (/3,rmin)} . (29) 
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Similarly, we can also give the optimal decision parameters M^; d^; fj,^, fj,2 ,..., such 
that 

{13, Tmax) = min {’Jf (/3, rmax)} • (30) 

Furthermore, we can get the optimal decision parameters M^; such 

that 

L'^ (/3) = min{’Jf (/3,rmax) - ^ (/3,rmin)}. (31) 


According to the above analysis, to design the supermarket model with different 
servers, it is seen from Equations ()29ll to ()31ll that here we provide two optimal crite- 
rions as follows: 

Criterion one: This supermarket model is better when choosing some parameters 
such that (/3, rmax) — {f3, rmin) I < for a given value > 0. 

Criterion two: This supermarket model is better when choosing some parameters 
such that L'^ (/3) < 82 for a given value <52 > 0. 

In general, the two optimal criterions can easily be implemented by means of the event- 
driven technique with performance simulation as well as perturbation realization, e.g., see 
Cao [6] and Xia and Cao |46] for more details. 

6 Performance Simulation 

In this section, we provide three simulation experiments whose purpose is to simply discuss 
how the expected queue length of each server depends on some key parameters: The choice 
number d, the service rate vector /i and the probability vector g of individual preference 
in the supermarket model with different servers. 

In the three simulation experiments, we take the server number M = 10 and the arrival 
rate A = 10. 

Experiment one: In the supermarket model with different servers, we take that the 
choice number d = 2; the service rates of the 10 servers are listed as fii = 1.1, ^2 = 1-2, 

= 1.3, fi4 = 1.4, /is = 1.5, He = 1.6, H7 = 1-7, /is = 1-8, /ig = 1-9 and gw = 2.0, 
respectively; the probabilities of individual preference for the 10 servers are given by 
gi = 0.10, 52 = 0.20, 53 = 0.30, 54 = 0.05, 55 = 0.05, ge = 0.02, 57 = 0.10, gs = 0.03, 
59 = 0.10 and 510 = 0.05, respectively. We simulate the expected queueing length for each 
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Table 2: The expected queue lengths in the 10 servers 


Server number 

Expected queue lengths 


One 

0.6834 


Two 

0.9454 


Three 

1.0440 


Four 

0.4318 


Five 

0.4234 


Six 

0.2894 


Seven 

0.4864 


Eight 

0.2793 


Nine 

0.4319 


Ten 

0.2640 



server by using the routine selection function 


1 + ^ 

Mx) = ,, _ _ , i = l,2, 


M r 

E 

1=1 


, 10 . 


1 + 

93 


The experimented results are shown in Table 1. 

Experiment two: This experiment takes the different parameters; only the 10 service 
rates, from that in Experiment one. That is, the choice number d = 2; the service rates 
of the 10 servers are listed as = 1, /U 2 = 2, = 6 , //4 = 8 , ^5 = 10, fiQ = 16, fij = 17, 
fig = 18, /Ug = 25 and nio = 26, respectively; the probabilities of individual preference 
for the 10 servers are given by gi = 0.10, g 2 = 0.20, gg = 0.30, (74 = 0.05, g^ = 0.05, 
g^ = 0.02, <77 = 0.10, <78 = 0.03, gg = 0.10 and <710 = 0.05, respectively. We still simulate 
the expected queueing length for each server by using the routine selection function 

1 + ^ 

^i{x) = — --, i = l,2,...,10. 

V [1 + ^ 

The experimented results are shown in Table 2. It is seen from Tables 1 and 2 that the 
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Table 3: The expected queue lengths in the 10 servers 


Server number 

Expected queue lengths 


One 

0.3459 


Two 

0.1656 


Three 

0.0274 


Four 

0.0158 


Five 

0.0105 


Six 

0.0042 


Seven 

0.0038 


Eight 

0.0034 


Nine 

0.0018 


Ten 

0.0017 



expected queue lengths of the M servers decrease, as the service rates of some servers 
increase. 

Experiment three: Comparing with Experiments one and two, this experiment takes 
more different parameters. We take that the choice number d = 3; the service rates of the 
10 servers are listed as = 1, //2 = 3, ns = 3, fi 4 = 6, = Q, hq = 6, fij = 6, = 9; 

/ig = 9 and = 15, respectively; the probabilities of individual preference for the 10 
servers are given by gi = 0.05, g 2 = 0.20, 53 = 0.30, <74 = 0.03, g^ = 0.05, g^ = 0.10, 
gj = 0.10, gs = 0.05, gg = 0.02 and <710 = 0.10, respectively. We simulate the expected 
queueing length for each server by using the routine selection function 

1 + ^ 

A*(x) = ^g- — -> i = l,2,...,10. 

E [l + 

The experimented results are shown in Table 3. It is seen from Tables 1, 2 and 3 that 
the expected queue lengths of the M servers decrease largely, as the choice number d 
changes from 2 to 3. Therefore, “the power of two choices” is still kept well in the study 
of supermarket models with different servers. 
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Table 4: The expected queue lengths in the 10 servers 


Server number 

Expected queue lengths 


One 

0.3447 


Two 

0.0580 


Three 

0.8598 


Four 

0.0265 


Five 

0.0265 


Six 

0.0266 


Seven 

0.0265 


Eight 

0.0126 


Nine 

0.0127 


Ten 

0.0048 



7 Concluding Remarks 

In this paper, we provide a novel method for analyzing the supermarket model with 
different servers through a multi-dimensional continuous-time Markov reward process, 
and develop an event-driven technique both for computing the mean of the random reward 
function in a finite time interval and for calculating the mean of the discounted random 
reward function in an infinite time interval. We indicate that the event-driven technique 
are useful in the study of supermarket models with different servers, and more generally, in 
the analysis of large-scale Markov reward processes. Notice that the supermarket model 
with different servers is an important tool to set up some basic relations between the 
system performance and the job routing rule, thus it can also help to design reasonable 
architecture to improve the performance and to balance the load in this supermarket 
model. 

This paper provides a clear picture for how to use the event-driven technique to analyze 
multi-dimensional continuous-time Markov reward processes, which leads to performance 
analysis of the supermarket model with different servers. We illustrate that this picture is 
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organized as three key parts: (1) Constructing a routine selection mechanism that depends 
on the queue lengths, on the service rates, on the probability of individual preference and 
so forth. (2) From the state jump points of the continuous-time Markov reward process, 
we set up some segmented stochastic integrals of the random reward function by means 
of an event-driven technique. Based on this, we compute the mean of the random reward 
function in a finite time interval, and also calculate the mean of the discounted random 
reward function in an infinite time interval. Therefore, the results of this paper give new 
highlight on understanding influence of the different servers on designing the routine se¬ 
lection mechanism and on performance computation of more general supermarket models. 
Along such a line, there are a number of interesting directions for potential future research, 
for example, 

• analyzing non-Poisson inputs such as, renewal processes; and discussing non-exponential 
service time distributions, for example, general distributions, matrix-exponential dis¬ 
tributions and heavy-tailed distributions; 

• studying how to design a new routine selection mechanism with respect to key ran¬ 
dom factors, such as, the least workload, and the subjective behavior of customers; 

• developing effective algorithms both for computing the means of the random reward 
functions and for solving the optimal problems in the study of supermarket models 
with different servers; and 

• The event-driven technique is further developed for discussing the sample paths of 
continuous-time Markov reward processes, thus the results given in this paper may 
be very useful for performance simulation of more general supermarket models with 
different servers. 

Up to now, we believe that a larger gap exists when dealing with either non-Poisson 
inputs or non-exponential service times in supermarket models with different servers, 
because the event-driven technique needs be established for being able to deal with more 
general Markov reward processes. 
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